Rigid Structure from Video

This thesis addresses two problems in content-based video analysis: the segmentation of a 2D rigid moving object; and the inference of 3D rigid structure. In Part I, we consider motion segmentation. Existing methods often fail to detect the motions of low textured regions and regions moving against low contrast background. We describe an algorithm to segment low textured moving objects that move against a low contrast background. Our approach has two distinguishing features. Our algorithm processes all frames available, as needed, while the majority of current motion segmentation methods use only two or three consecutive images. Second, we model explicitly the oclusion of the background by the moving object and recover the shape of the moving object directly from the image intensity values. This contrasts with other approaches that deal with low textured scenes by attempting to smooth out a sparse set of motion measurements. In Part II, we develop a factorization method that recovers the 3D shape and 3D motion of rigid moving objects whose surface shape is parameterized by a nite set of parameters. Our approach induces a parametric model for the 2D motion of the brightness pattern in the image plane. To estimate the 3D shape and 3D motion parameters from the 2D motion parameters, we introduce the surface-based rank 1 factorization algorithm. Our method uses an appropriate linear subspace projection that leads to the factorization of a matrix that is rank 1 in a noiseless situation. This allows the use of fast iterative algorithms to compute the 3D structure that best ts the data. We track regions where the 2D motion in the image plane is described by a single set of parameters. Our method contrasts with the original factorization method that required tracking a large number of pointwise features, in general a di cult task, and the factorization of a rank 3 matrix. We apply to both problems Maximum Likelihood estimation techniques. Illustrative examples show the good quality of our algorithms.